Scalable load balancing in the presence of heterogeneous servers

Heterogeneity is becoming increasingly ubiquitous in modern large-scale computer systems. Developing good load balancing policies for systems whose resources have varying speeds crucial achieving low response times. Indeed, how best to dispatch jobs servers a classical and well-studied problem the queueing literature. Yet bulk of existing work on assumes homogeneous servers; unfortunately, that perform well setting can cause unacceptably poor performance heterogeneous We adapt “power-of-d” versions both Join-the-Idle-Queue Join-the-Shortest-Queue design two corresponding families heterogeneity-aware dispatching policies, each which parameterized by pair routing probabilities. Unlike their heterogeneity-unaware counterparts, our use server speed information when choosing query probabilistically deciding where (among queried servers) jobs. Both policy are analytically tractable: mean time queue length distribution analyses exact as number approaches infinity, under standard assumptions. Furthermore, achieve maximal stability outperform well-known rules – including such Shortest-Expected-Delay with respect time.